In this document, we intend to investigate the following key questions, assuming a fixed array of \(10^6\) SNPs and a quantitative trait in which SNP effect sizes follow a normal distribution:
In this section, we look at the average number of significant SNPs, the average proportion of these significant SNPs that have association estimates more extreme than their true effect size and the average MSE of significant SNPs at two different thresholds; the common genome-wide significance threshold of \(5 \times 10^{-8}\) and a higher threshold of \(5 \times 10^{-4}\). We consider these properties under certain combinations of values for the following parameters:
n_samplesh2prop_effectSThe 24 different combinations that we will investigate throughout this document are detailed below:
| Scenario | n_samples | h2 | prop_effect | S |
|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 |
| 2 | 300,000 | 0.3 | 0.010 | -1 |
| 3 | 30,000 | 0.8 | 0.010 | -1 |
| 4 | 300,000 | 0.8 | 0.010 | -1 |
| 5 | 30,000 | 0.3 | 0.001 | -1 |
| 6 | 300,000 | 0.3 | 0.001 | -1 |
| 7 | 30,000 | 0.8 | 0.001 | -1 |
| 8 | 300,000 | 0.8 | 0.001 | -1 |
| 9 | 30,000 | 0.3 | 0.010 | 0 |
| 10 | 300,000 | 0.3 | 0.010 | 0 |
| 11 | 30,000 | 0.8 | 0.010 | 0 |
| 12 | 300,000 | 0.8 | 0.010 | 0 |
| Scenario | n_samples | h2 | prop_effect | S |
|---|---|---|---|---|
| 13 | 30,000 | 0.3 | 0.001 | 0 |
| 14 | 300,000 | 0.3 | 0.001 | 0 |
| 15 | 30,000 | 0.8 | 0.001 | 0 |
| 16 | 300,000 | 0.8 | 0.001 | 0 |
| 17 | 30,000 | 0.3 | 0.010 | 1 |
| 18 | 300,000 | 0.3 | 0.010 | 1 |
| 19 | 30,000 | 0.8 | 0.010 | 1 |
| 20 | 300,000 | 0.8 | 0.010 | 1 |
| 21 | 30,000 | 0.3 | 0.001 | 1 |
| 22 | 300,000 | 0.3 | 0.001 | 1 |
| 23 | 30,000 | 0.8 | 0.001 | 1 |
| 24 | 300,000 | 0.8 | 0.001 | 1 |
\(~\) \(~\) \(~\)
Running the code provided in nsig_prop_bias_100sim.R, we obtain the following results. Note that prop_x refers to the proportion of significant SNPs which have been found to be significantly overestimated, i.e. those SNPs in which \(\left| \hat\beta_{i} \right| > \left| \beta_{i} \right| + 1.96\cdot\sigma_{i}\).
| Scenario | n_samples | h2 | prop_effect | S | n_sig 5e-8 | prop_bias 5e-8 | prop_x 5e-8 | mse 5e-8 | n_sig 5e-4 | prop_bias 5e-4 | prop_x 5e-4 | mse 5e-4 | sd(n_sig) 5e-8 | sd(prop_bias) 5e-8 | sd(prop_x) 5e-8 | sd(mse) 5e-8 | sd(n_sig) 5e-4 | sd(prop_bias) 5e-4 | sd(prop_x) 5e-4 | sd(mse) 5e-4 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | 0.70 | 1.0000 | 0.9000 | 0.001573 | 610.99 | 0.9996 | 0.9165 | 0.001919 | 0.745 | 0.0000 | 0.2950 | 0.001234 | 23.070 | 0.0008 | 0.0108 | 0.000105 |
| 2 | 300,000 | 0.3 | 0.010 | -1 | 848.63 | 0.7619 | 0.0895 | 0.000022 | 3201.35 | 0.7461 | 0.2083 | 0.000049 | 18.145 | 0.0142 | 0.0103 | 0.000002 | 47.967 | 0.0070 | 0.0071 | 0.000003 |
| 3 | 30,000 | 0.8 | 0.010 | -1 | 31.85 | 0.9804 | 0.4020 | 0.000598 | 1089.98 | 0.9597 | 0.5647 | 0.001194 | 5.208 | 0.0280 | 0.0834 | 0.000198 | 35.158 | 0.0053 | 0.0136 | 0.000060 |
| 4 | 300,000 | 0.8 | 0.010 | -1 | 2760.68 | 0.6284 | 0.0484 | 0.000017 | 5349.44 | 0.6393 | 0.1305 | 0.000035 | 30.091 | 0.0084 | 0.0038 | 0.000001 | 43.729 | 0.0069 | 0.0045 | 0.000002 |
| 5 | 30,000 | 0.3 | 0.001 | -1 | 86.90 | 0.7591 | 0.0892 | 0.000214 | 772.12 | 0.8957 | 0.6724 | 0.001474 | 6.317 | 0.0437 | 0.0290 | 0.000054 | 25.913 | 0.0090 | 0.0140 | 0.000088 |
| 6 | 300,000 | 0.3 | 0.001 | -1 | 568.70 | 0.5509 | 0.0330 | 0.000016 | 1211.96 | 0.7303 | 0.4300 | 0.000099 | 14.074 | 0.0225 | 0.0078 | 0.000002 | 24.787 | 0.0128 | 0.0127 | 0.000006 |
| 7 | 30,000 | 0.8 | 0.001 | -1 | 276.26 | 0.6273 | 0.0464 | 0.000168 | 986.59 | 0.8043 | 0.5276 | 0.001198 | 10.413 | 0.0271 | 0.0123 | 0.000027 | 26.682 | 0.0117 | 0.0138 | 0.000075 |
| 8 | 300,000 | 0.8 | 0.001 | -1 | 727.10 | 0.5257 | 0.0287 | 0.000016 | 1326.17 | 0.7068 | 0.3971 | 0.000093 | 12.491 | 0.0171 | 0.0061 | 0.000001 | 24.072 | 0.0127 | 0.0105 | 0.000005 |
| 9 | 30,000 | 0.3 | 0.010 | 0 | 1.45 | 1.0000 | 0.8610 | 0.001661 | 627.16 | 0.9985 | 0.8888 | 0.001825 | 1.298 | 0.0000 | 0.2745 | 0.005559 | 25.180 | 0.0015 | 0.0128 | 0.000090 |
| 10 | 300,000 | 0.3 | 0.010 | 0 | 882.45 | 0.7297 | 0.0807 | 0.000012 | 3054.78 | 0.7427 | 0.2176 | 0.000046 | 18.435 | 0.0152 | 0.0098 | 0.000001 | 45.422 | 0.0070 | 0.0066 | 0.000002 |
| 11 | 30,000 | 0.8 | 0.010 | 0 | 48.06 | 0.9509 | 0.2908 | 0.000245 | 1110.51 | 0.9407 | 0.5435 | 0.001091 | 6.350 | 0.0301 | 0.0645 | 0.000050 | 29.205 | 0.0068 | 0.0123 | 0.000060 |
| 12 | 300,000 | 0.8 | 0.010 | 0 | 2586.78 | 0.6204 | 0.0477 | 0.000011 | 5032.06 | 0.6455 | 0.1375 | 0.000032 | 32.771 | 0.0096 | 0.0038 | 0.000000 | 45.218 | 0.0062 | 0.0046 | 0.000002 |
| 13 | 30,000 | 0.3 | 0.001 | 0 | 88.32 | 0.7254 | 0.0774 | 0.000116 | 755.09 | 0.8953 | 0.6820 | 0.001491 | 6.377 | 0.0480 | 0.0279 | 0.000025 | 22.799 | 0.0097 | 0.0131 | 0.000077 |
| 14 | 300,000 | 0.3 | 0.001 | 0 | 531.27 | 0.5560 | 0.0334 | 0.000012 | 1183.18 | 0.7402 | 0.4407 | 0.000100 | 12.431 | 0.0215 | 0.0074 | 0.000001 | 27.843 | 0.0121 | 0.0091 | 0.000006 |
| 15 | 30,000 | 0.8 | 0.001 | 0 | 257.44 | 0.6234 | 0.0473 | 0.000106 | 953.63 | 0.8130 | 0.5449 | 0.001202 | 8.936 | 0.0276 | 0.0120 | 0.000013 | 23.262 | 0.0100 | 0.0115 | 0.000070 |
| 16 | 300,000 | 0.8 | 0.001 | 0 | 691.46 | 0.5268 | 0.0301 | 0.000013 | 1301.33 | 0.7114 | 0.4040 | 0.000092 | 13.526 | 0.0185 | 0.0066 | 0.000001 | 29.469 | 0.0110 | 0.0123 | 0.000005 |
| 17 | 30,000 | 0.3 | 0.010 | 1 | 2.55 | 0.9941 | 0.7339 | 0.000610 | 641.18 | 0.9966 | 0.8643 | 0.001777 | 1.623 | 0.0405 | 0.3034 | 0.000674 | 25.497 | 0.0022 | 0.0126 | 0.000107 |
| 18 | 300,000 | 0.3 | 0.010 | 1 | 919.28 | 0.7031 | 0.0706 | 0.000010 | 2902.71 | 0.7313 | 0.2228 | 0.000046 | 15.799 | 0.0142 | 0.0079 | 0.000001 | 38.290 | 0.0086 | 0.0073 | 0.000003 |
| 19 | 30,000 | 0.8 | 0.010 | 1 | 68.13 | 0.9178 | 0.2410 | 0.000198 | 1141.84 | 0.9208 | 0.5196 | 0.001047 | 7.795 | 0.0370 | 0.0486 | 0.000087 | 29.135 | 0.0077 | 0.0144 | 0.000063 |
| 20 | 300,000 | 0.8 | 0.010 | 1 | 2433.27 | 0.6105 | 0.0462 | 0.000009 | 4634.50 | 0.6464 | 0.1461 | 0.000032 | 30.705 | 0.0107 | 0.0047 | 0.000000 | 48.828 | 0.0078 | 0.0046 | 0.000002 |
| 21 | 30,000 | 0.3 | 0.001 | 1 | 93.15 | 0.7056 | 0.0684 | 0.000096 | 742.12 | 0.8944 | 0.6949 | 0.001509 | 5.960 | 0.0428 | 0.0273 | 0.000015 | 23.899 | 0.0102 | 0.0140 | 0.000080 |
| 22 | 300,000 | 0.3 | 0.001 | 1 | 482.94 | 0.5516 | 0.0329 | 0.000009 | 1124.74 | 0.7510 | 0.4635 | 0.000103 | 12.742 | 0.0244 | 0.0072 | 0.000001 | 24.808 | 0.0116 | 0.0117 | 0.000006 |
| 23 | 30,000 | 0.8 | 0.001 | 1 | 244.29 | 0.6120 | 0.0448 | 0.000089 | 917.81 | 0.8199 | 0.5673 | 0.001249 | 9.237 | 0.0272 | 0.0142 | 0.000010 | 25.125 | 0.0110 | 0.0137 | 0.000084 |
| 24 | 300,000 | 0.8 | 0.001 | 1 | 632.74 | 0.5336 | 0.0294 | 0.000010 | 1235.79 | 0.7201 | 0.4192 | 0.000094 | 14.730 | 0.0194 | 0.0071 | 0.000001 | 25.984 | 0.0121 | 0.0112 | 0.000005 |
\(~\) \(~\) \(~\)
★ It is important to note here that for scenarios 1, 9 and 17, very few significant SNPs are detected on average. In some instances, we may even find that no SNPs are deemed significant at a threshold of \(5 \times 10^{-8}\). We must keep this observation in mind going forward as we investigate the performance of methods under these three scenarios.
For both thresholds, the average number of significant SNPs increases as sample size increases, as expected. It also increases with heritability. However, the effect of changing prop_effect is more interesting. Decreasing the proportion of effect SNPs from 0.01 to 0.001 results in the number of significant SNPs increasing for a sample size of 30,000 while we witness the number of SNPs passing the genome-wide significance threshold decreasing for a larger sample size of 300,000.
Furthermore, increasing sample size and increasing heritability from 0.3 to 0.8 all tend to decrease the fraction of significant SNPs whose estimates are more extreme than their true effect size. Decreasing polygenicity from 0.01 to 0.001 also has this same effect at a significance threshold of \(5 \times 10^{-8}\).
As sample size increases from 30,000 to 300,000 in all scenarios, prop_x, the proportion of significant SNPs which have been found to be significantly overestimated, decreases. This is a strong indicator that as the value of n_samples increases, we expect the bias induced by Winner’s Curse to be less of a problem among significant SNPs. However, this could also be due to the fact that as sample size increases, the number of significant SNPs passing the genome-wide significance threshold of 5e-8 also increases.
In order to gain a better insight into the information detailed in the above table, we simulate a single set of GWAS summary statistics and plot \(z\) vs \(\text{bias}\) in which \(\text{bias} = \hat\beta - \beta\) for each of the 24 different scenarios. On all figures, the bright red line corresponds to the significance threshold of \(5 \times 10^{-8}\) while the darker red line relates to \(5 \times 10^{-4}\). The points corresponding to SNPs which are significantly overestimated and are significant at a threshold of \(5 \times 10^{-4}\) are coloured in navy .
Using the code detailed in norm_5e-8_20sim.R and a total of 20 simulations, we evaluated six different Winner’s Curse methods across each of the 24 scenarios using the following three bias evaluation metrics:
flbmserel_mseNote: All averages are obtained over only those simulations in which at least one significant SNP was detected.
Firstly, the fraction of \(n\) significant SNPs in which their association estimates have been improved due to method implementation may be mathematically described as: \[\frac{1}{n} \; \sum_{i=1}^{n}\mathbb{I} \left\{ \left| \hat\beta_i - \beta_i \right| > \left|\hat\beta_{\text{adj,}i} - \beta_i\right| \right\},\]in which \(\left| \frac{\hat\beta_i}{\hat\sigma_i} \right| > Z_{\frac{\alpha}{2}}\) for all \(i = 1,...,n\), where \(\hat\beta_i\) is the estimated naive effect size of SNP \(i\), \(\beta_i\) is its true effect size and \(\hat\beta_{\text{adj,}i}\) is its new effect size estimate obtained as a result of application of the Winner’s Curse adjustment method of interest. The significance threshold is represented by \(\alpha\).
Using the same notation, the average MSE over \(n\) significant SNPs is defined as: \[\frac{1}{n} \sum^n_{i=1} (\hat\beta_i - \beta_i)^2.\] Thus, using the above, we may formally define the change in average MSE of significant SNPs as: \[\frac{1}{n} \sum^n_{i=1} (\hat\beta_{\text{adj,}i} - \beta_i)^2 - \frac{1}{n} \sum^n_{i=1} (\hat\beta_i - \beta_i)^2\] and the relative change in average MSE of significant SNPs as: \[\frac{\frac{1}{n} \sum^n_{i=1} (\hat\beta_{\text{adj,}i} - \beta_i)^2 - \frac{1}{n} \sum^n_{i=1} (\hat\beta_i - \beta_i)^2}{\frac{1}{n} \sum^n_{i=1} (\hat\beta_i - \beta_i)^2}.\]
\(~\)
Results of the simulations are plotted. Error bars are also included in the plots. These figures allow us to see more clearly the scenarios in which it would be beneficial to apply a Winner’s Curse correction method and also, provide us with a better indication of which method we should use.
<<<<<<< HEADA replication method which selects significant SNPs from a discovery GWAS and then uses a replication GWAS of the same size to obtain association estimates for these SNPs is also included in the plots. This can be viewed as acting as a form of benchmark for the other methods.
Summary of results for flb contained in norm_5e-8_20sim.csv:
Summary of results for flb contained in norm_5e-8_20sim.csv:
| Scenario | n_samples | h2 | prop_effect | S | EB | FIQT | BR | cl1 | cl2 | cl3 | rep | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | 0.9000 | 0.9000 | 0.8500 | 0.7500 | 1.0000 | 0.9000 | 1.0000 | |||||
| 2 | 300,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD 0.6396 | 0.4446 | 0.6034 | 0.5289 | 0.4974 | 0.5095 | 0.5612 ======= 0.6417 | 0.4368 | 0.6028 | 0.5320 | 0.5017 | 0.5134 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 3 | 30,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD 0.9140 | 0.7580 | 0.7802 | 0.6152 | 0.7338 | 0.6657 | 0.8166 ======= 0.8891 | 0.7522 | 0.7579 | 0.5865 | 0.6850 | 0.6306 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 4 | 300,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD 0.5673 | 0.2811 | 0.5319 | 0.5174 | 0.4865 | 0.5014 | 0.5138 ======= 0.5654 | 0.2814 | 0.5327 | 0.5148 | 0.4845 | 0.4990 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 5 | 30,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD 0.6312 | 0.3257 | 0.4595 | 0.5253 | 0.4911 | 0.5068 | 0.5631 ======= 0.6353 | 0.3602 | 0.4889 | 0.5328 | 0.5018 | 0.5134 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 6 | 300,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD 0.4926 | 0.1340 | 0.2759 | 0.5097 | 0.4778 | 0.5017 | 0.5002 ======= 0.4805 | 0.1341 | 0.2665 | 0.4993 | 0.4725 | 0.4932 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 7 | 30,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD 0.5230 | 0.2248 | 0.2974 | 0.5213 | 0.4886 | 0.5037 | 0.5118 ======= 0.5245 | 0.2213 | 0.3009 | 0.5152 | 0.4841 | 0.4963 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 8 | 300,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD 0.4757 | 0.1053 | 0.3009 | 0.5028 | 0.4711 | 0.4894 | 0.5082 ======= 0.4788 | 0.0991 | 0.3036 | 0.5034 | 0.4772 | 0.4906 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 9 | 30,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD 0.9062 ======= 0.9375 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.9375 | <<<<<<< HEAD 0.8438 | 0.9062 ======= 0.8750 | 0.7500 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.9375 | <<<<<<< HEAD 0.9062 | 0.9375 ======= 0.8125 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||||
| 10 | 300,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD 0.6212 | 0.4058 | 0.5772 | 0.5291 | 0.4977 | 0.5120 | 0.5466 ======= 0.6214 | 0.4054 | 0.5806 | 0.5349 | 0.5017 | 0.5153 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 11 | 30,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD 0.8435 | 0.6979 | 0.7417 | 0.5920 | 0.6757 | 0.6251 | 0.7557 ======= 0.8396 | 0.6581 | 0.7120 | 0.5782 | 0.6356 | 0.6007 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 12 | 300,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD 0.5589 | 0.2708 | 0.5246 | 0.5120 | 0.4817 | 0.4958 | 0.5092 ======= 0.5552 | 0.2711 | 0.5216 | 0.5117 | 0.4817 | 0.4968 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 13 | 30,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD 0.6006 | 0.3048 | 0.4487 | 0.5061 | 0.4820 | 0.4932 | 0.5441 ======= 0.5918 | 0.3020 | 0.4396 | 0.5192 | 0.4835 | 0.4918 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 14 | 300,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD 0.4862 | 0.1390 | 0.2821 | 0.5043 | 0.4742 | 0.4964 | 0.4942 ======= 0.4869 | 0.1382 | 0.2834 | 0.5100 | 0.4767 | 0.5010 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 15 | 30,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD 0.5159 | 0.2013 | 0.2860 | 0.5130 | 0.4872 | 0.5008 | 0.5100 ======= 0.5150 | 0.2139 | 0.2879 | 0.5155 | 0.4865 | 0.4987 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 16 | 300,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD 0.4635 | 0.1089 | 0.3005 ======= 0.4587 | 0.1091 | 0.3000 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.5023 | <<<<<<< HEAD 0.4653 ======= 0.4582 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.4833 | 0.5002 | |||
| 17 | 30,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD 0.7792 | 0.8292 | 0.8083 | 0.6670 | 0.8708 | 0.7620 | 0.9358 ======= 0.8546 | 0.8685 | 0.8296 | 0.6102 | 0.8898 | 0.8370 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 18 | 300,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD 0.5961 | 0.3647 | 0.5290 | 0.5219 | 0.4902 | 0.5033 | 0.5399 ======= 0.5970 | 0.3661 | 0.5346 | 0.5239 | 0.4910 | 0.5029 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 19 | 30,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD 0.7829 | 0.5712 | 0.6422 | 0.5588 | 0.5879 | 0.5679 | 0.6950 ======= 0.7977 | 0.6161 | 0.6807 | 0.5506 | 0.5915 | 0.5588 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 20 | 300,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD 0.5510 | 0.2573 | 0.5109 | 0.5095 | 0.4803 | 0.4957 | 0.5121 ======= 0.5541 | 0.2583 | 0.5164 | 0.5134 | 0.4855 | 0.4988 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 21 | 30,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD 0.5875 | 0.2870 | 0.4261 | 0.5109 | 0.4843 | 0.4958 | 0.5459 ======= 0.5839 | 0.2741 | 0.4091 | 0.5132 | 0.4809 | 0.4945 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 22 | 300,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD 0.4799 | 0.1429 | 0.2779 | 0.5050 | 0.4704 | 0.4960 | 0.4955 ======= 0.4760 | 0.1508 | 0.2848 | 0.5102 | 0.4808 | 0.5015 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 23 | 30,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 0.4990 | 0.1928 | 0.2653 | 0.5044 | 0.4802 | 0.4915 | 0.5176 ======= 0.5052 | 0.1824 | 0.2652 | 0.5100 | 0.4827 | 0.4982 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 24 | 300,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 0.4653 | 0.1098 | 0.3093 | 0.5043 | 0.4576 | 0.4790 | 0.5022 ======= 0.4579 | 0.1148 | 0.3095 | 0.5037 | 0.4628 | 0.4786 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
\(~\) \(~\) \(~\) \(~\)
Fraction of significant SNPs with improved association estimates due to method implementation, using a significance threshold of \(5 \times 10^{-8}\):
<<<<<<< HEAD \(~\) \(~\) \(~\) \(~\)
Summary of results for mse contained in norm_5e-8_20sim.csv:
\(~\) \(~\) \(~\) \(~\)
Summary of results for mse contained in norm_5e-8_20sim.csv:
| Scenario | n_samples | h2 | prop_effect | S | EB | FIQT | BR | cl1 | cl2 | cl3 | rep | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.001739 | -0.001747 | -0.001756 | -0.001440 | -0.001631 | -0.001592 | -0.001604 ======= -0.001036 | -0.001113 | -0.001070 | -0.000810 | -0.001042 | -0.000971 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 2 | 300,000 | 0.3 | 0.010 | -1 | -0.000006 | 0.000001 | -0.000004 | 0.000052 | 0.000023 <<<<<<< HEAD | 0.000035 | -0.000005 ======= | 0.000035 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||||
| 3 | 30,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.000435 | -0.000453 | -0.000464 | 0.000158 | -0.000327 | -0.000139 | -0.000442 ======= -0.000397 | -0.000382 | -0.000399 | 0.000340 | -0.000251 | -0.000016 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 4 | 300,000 | 0.8 | 0.010 | -1 | -0.000002 | 0.000005 | 0.000000 | 0.000035 | 0.000020 | <<<<<<< HEAD 0.000026 | -0.000001 ======= 0.000025 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||||
| 5 | 30,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD -0.000032 | 0.000130 | 0.000073 | 0.000599 | 0.000245 | 0.000391 | -0.000040 ======= -0.000052 | 0.000128 | 0.000056 | 0.000553 | 0.000246 | 0.000371 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 6 | 300,000 | 0.3 | 0.001 | -1 | 0.000001 | 0.000007 | 0.000013 | 0.000014 | 0.000009 | <<<<<<< HEAD 0.000012 | 0.000000 ======= 0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||||
| 7 | 30,000 | 0.8 | 0.001 | -1 | -0.000001 | <<<<<<< HEAD 0.000134 | 0.000163 | 0.000322 | 0.000192 | 0.000243 | -0.000015 ======= 0.000118 | 0.000139 | 0.000356 | 0.000198 | 0.000262 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |
| 8 | 300,000 | 0.8 | 0.001 | -1 | 0.000001 | 0.000003 | 0.000007 | 0.000008 | <<<<<<< HEAD 0.000023 | 0.000011 | 0.000000 ======= 0.000022 | 0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||||
| 9 | 30,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.000742 | -0.000767 | -0.000719 | -0.000592 | -0.000750 | -0.000703 | -0.000670 ======= -0.000482 | -0.000506 | -0.000459 | -0.000173 | -0.000431 | -0.000342 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 10 | 300,000 | 0.3 | 0.010 | 0 | -0.000003 | 0.000001 | -0.000002 | 0.000026 | 0.000011 | 0.000017 <<<<<<< HEAD | -0.000003 ======= >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||||
| 11 | 30,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.000144 | -0.000137 | -0.000149 | 0.000184 | -0.000065 | 0.000035 | -0.000153 ======= -0.000153 | -0.000135 | -0.000146 | 0.000206 | -0.000055 | 0.000050 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 12 | 300,000 | 0.8 | 0.010 | 0 | -0.000001 | 0.000002 | 0.000000 | 0.000021 | 0.000012 | 0.000015 | -0.000001 | |||||
| 13 | 30,000 | 0.3 | 0.001 | 0 | -0.000021 <<<<<<< HEAD | 0.000073 | 0.000038 | 0.000292 | 0.000124 | 0.000193 | -0.000028 ======= | 0.000073 | 0.000046 | 0.000286 | 0.000129 | 0.000193 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 14 | 300,000 | 0.3 | 0.001 | 0 | 0.000000 | 0.000005 | 0.000009 | 0.000014 | 0.000009 | 0.000011 <<<<<<< HEAD | 0.000000 ======= >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||||
| 15 | 30,000 | 0.8 | 0.001 | 0 | -0.000003 | <<<<<<< HEAD 0.000069 | 0.000105 | 0.000232 | 0.000121 | 0.000166 | -0.000012 ======= 0.000064 | 0.000097 | 0.000205 | 0.000108 | 0.000148 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |
| 16 | 300,000 | 0.8 | 0.001 | 0 | 0.000001 | 0.000004 | 0.000006 | <<<<<<< HEAD 0.000011 | 0.000026 | 0.000014 | 0.000000 ======= 0.000013 | 0.000030 | 0.000015 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||
| 17 | 30,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.000500 | -0.000565 | -0.000532 | -0.000297 | -0.000526 | -0.000449 | -0.000609 ======= -0.000416 | -0.000426 | -0.000399 | -0.000083 | -0.000398 | -0.000282 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 18 | 300,000 | 0.3 | 0.010 | 1 | -0.000002 | 0.000002 | 0.000000 | 0.000023 | 0.000011 | 0.000015 | -0.000002 | |||||
| 19 | 30,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.000113 | -0.000083 | -0.000102 | 0.000214 | -0.000011 ======= -0.000112 | -0.000085 | -0.000101 | 0.000236 | -0.000002 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.000095 | -0.000125 | |
| 20 | 300,000 | 0.8 | 0.010 | 1 | -0.000001 | 0.000002 | 0.000000 | 0.000016 | 0.000009 | 0.000012 | -0.000001 | |||||
| 21 | 30,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD -0.000015 | 0.000072 | 0.000046 | 0.000246 | 0.000109 | 0.000165 | -0.000026 ======= -0.000010 | 0.000070 | 0.000045 | 0.000233 | 0.000107 | 0.000158 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 22 | 300,000 | 0.3 | 0.001 | 1 | 0.000000 | 0.000003 | 0.000007 | 0.000010 | 0.000007 | 0.000008 <<<<<<< HEAD | 0.000000 ======= >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||||
| 23 | 30,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 0.000003 ======= 0.000001 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.000056 | <<<<<<< HEAD 0.000103 | 0.000156 | 0.000089 | 0.000116 | -0.000005 ======= 0.000099 | 0.000160 | 0.000088 | 0.000116 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||
| 24 | 300,000 | 0.8 | 0.001 | 1 | 0.000001 | 0.000002 | 0.000005 | 0.000008 | <<<<<<< HEAD 0.000044 | 0.000016 | 0.000000 ======= 0.000037 | 0.000014 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
\(~\) \(~\) \(~\) \(~\)
Change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\):
<<<<<<< HEAD\(~\) \(~\) \(~\) \(~\)
Summary of results for rel_mse contained in norm_5e-8_20sim.csv:
\(~\) \(~\) \(~\) \(~\)
Summary of results for rel_mse contained in norm_5e-8_20sim.csv:
| Scenario | n_samples | h2 | prop_effect | S | EB | FIQT | BR | cl1 | cl2 | cl3 | rep | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.8659 | -0.8976 | -0.8466 | -0.6782 | -0.8740 | -0.8233 | -0.8254 ======= -0.6604 | -0.7586 | -0.5970 | -0.0416 | -0.7533 | -0.4873 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 2 | 300,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.3143 | 0.0630 | -0.2040 | 2.5177 | 1.0966 | 1.6754 | -0.2480 ======= -0.2989 | 0.0632 | -0.1870 | 2.5004 | 1.0869 | 1.6622 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 3 | 30,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.7314 | -0.7086 | -0.7393 | 0.3578 | -0.5004 | -0.1665 | -0.7096 ======= -0.7280 | -0.6790 | -0.7147 | 0.6379 | -0.4443 | -0.0140 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 4 | 300,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.0977 | 0.2689 | -0.0143 | 2.0181 | 1.1567 | 1.5003 | -0.0817 ======= -0.0898 | 0.2845 | -0.0033 | 2.0018 | 1.1615 | 1.4939 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 5 | 30,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD -0.1375 | 0.7887 | 0.4699 | 3.3286 | 1.4118 | 2.2051 | -0.1428 ======= -0.1757 | 0.7489 | 0.3820 | 2.7333 | 1.2761 | 1.8716 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 6 | 300,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD 0.0349 | 0.4142 | 0.8520 | 0.9163 | 0.6300 | 0.7364 | -0.0251 ======= 0.0517 | 0.4211 | 0.8243 | 0.8484 | 0.5965 | 0.6886 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 7 | 30,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD -0.0032 | 0.7892 | 0.9577 | 1.8979 | 1.1281 | 1.4273 | -0.0683 ======= -0.0035 | 0.7616 | 0.8963 | 2.2455 | 1.2639 | 1.6585 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 8 | 300,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD 0.0638 | 0.2201 | 0.4378 | 0.5427 | 1.4651 | 0.6953 | 0.0091 ======= 0.0512 | 0.2099 | 0.4236 | 0.5236 | 1.4262 | 0.6775 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 9 | 30,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.5376 | -0.6701 | -0.4098 | 0.1175 | -0.6814 | -0.3631 | -0.6809 ======= -0.2439 | -0.4177 | -0.1850 | 0.7787 | -0.4338 | 0.0504 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 10 | 300,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.2770 | 0.0366 | -0.1718 | 2.2535 | 0.9473 | 1.4787 | -0.2795 ======= -0.2636 | 0.0651 | -0.1594 | 2.3332 | 1.0020 | 1.5453 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 11 | 30,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.6387 | -0.5912 | -0.6478 | 0.8678 | -0.2670 | 0.1896 | -0.6672 ======= -0.6696 | -0.5757 | -0.6260 | 0.9705 | -0.2137 | 0.2651 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 12 | 300,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.1012 | 0.1955 | -0.0384 | 1.9523 | 1.1193 | 1.4323 | -0.1073 ======= -0.0982 | 0.2001 | -0.0355 | 1.9867 | 1.0702 | 1.4377 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 13 | 30,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD -0.1800 | 0.6662 | 0.3534 | 2.5659 | 1.1128 | 1.7091 | -0.2209 ======= -0.1805 | 0.7452 | 0.4794 | 2.6822 | 1.2278 | 1.8205 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 14 | 300,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD 0.0303 | 0.4008 | 0.7867 | 1.2410 | 0.7627 | 0.9444 | -0.0137 ======= 0.0330 | 0.4196 | 0.7916 | 1.2425 | 0.7841 | 0.9560 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 15 | 30,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD -0.0099 | 0.6751 | 1.0223 | 2.2359 | 1.1727 | 1.6048 | -0.1075 ======= -0.0224 | 0.6125 | 0.9225 | 1.9473 | 1.0361 | 1.4060 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 16 | 300,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD 0.1443 | 0.3015 | 0.4888 | 0.8795 | 2.0278 | 1.0410 | -0.0314 ======= 0.1168 | 0.3167 | 0.5312 | 1.0189 | 2.4554 | 1.2144 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 17 | 30,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.6014 | -0.6888 | -0.6122 | 0.1929 | -0.6644 | -0.3321 | -0.8068 ======= -0.7281 | -0.7308 | -0.6273 | 0.3158 | -0.7049 | -0.3138 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 18 | 300,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.2023 | 0.2016 | -0.0206 | 2.2877 | 1.1015 | 1.5804 | -0.2169 ======= -0.1962 | 0.1977 | -0.0266 | 2.3515 | 1.1098 | 1.6144 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 19 | 30,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.5828 | -0.3981 | -0.5037 | 1.2294 | 0.0019 | 0.4995 | -0.6136 ======= -0.5992 | -0.4470 | -0.5346 | 1.2799 | -0.0018 | 0.5187 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 20 | 300,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.0734 | 0.2387 | 0.0130 | 1.8655 | 1.0276 | 1.3627 | -0.0955 ======= -0.0799 | 0.2242 | 0.0054 | 1.8330 | 1.0099 | 1.3388 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 21 | 30,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD -0.1503 | 0.7625 | 0.4876 | 2.5461 | 1.1361 | 1.7120 | -0.2582 ======= -0.0957 | 0.7492 | 0.4913 | 2.4301 | 1.1282 | 1.6581 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 22 | 300,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD 0.0506 | 0.4088 | 0.8656 | 1.1302 | 0.7106 | 0.8616 | -0.0083 ======= 0.0490 | 0.3649 | 0.8044 | 1.1342 | 0.7166 | 0.8661 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 23 | 30,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 0.0373 | 0.6623 | 1.2072 | 1.8365 | 1.0397 | 1.3586 | -0.0476 ======= 0.0113 | 0.6257 | 1.1099 | 1.7927 | 0.9795 | 1.3027 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 24 | 300,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 0.1362 | 0.2605 | 0.4774 | 0.8255 | 4.3116 | 1.5529 | -0.0373 ======= 0.1577 | 0.2511 | 0.4881 | 0.8140 | 3.6795 | 1.4035 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
\(~\) \(~\) \(~\) \(~\)
The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:
## EB FIQT BR cl1 cl2 cl3 rep
## -0.2086333 0.1416583 0.1754542 1.4553375 0.8488625 0.9951000 -0.2618500
The methods are ranked according to the results above in ascending order:
## EB FIQT BR cl1 cl2 cl3 rep
## 2 3 4 7 5 6 1
\(~\) \(~\) \(~\) \(~\)
Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\):
<<<<<<< HEADIt is worth noting from the above simulations how the Winner’s Curse methods tend to break down, or equivalently, no longer make improvements based on the third evaluation metric, rel_mse when the proportion of effect SNPs is 0.001. This is a measure of polygenicity. That said, the empirical Bayes method performs extremely similar to just taking a replication estimate when the replication sample is of the same size as the discovery GWAS, i.e. n_samples is equivalent in both data sets and as we will see later, the empirical Bayes will outperform the replication method when the replication GWAS sample size becomes smaller than that of the discovery, at this threshold of 5e-8.
On all occasions, the conditional likelihood methods are performing poorly compared to the other methods.
=======Similar to part 2 above, we use the code detailed in norm_5e-4_10sim.R with a total of 10 simulations in order to evaluate seven different Winner’s Curse methods across each of the 24 scenarios. In the following investigations, we will concentrate on the final bias evaluation metric, rel_mse.
\(~\) \(~\) \(~\) \(~\)
Summary of results for rel_mse contained in norm_5e-4_10sim.csv:
| Scenario | n_samples | h2 | prop_effect | S | EB | FIQT | BR | cl1 | cl2 | cl3 | rep | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.9516 | -0.9538 | -0.8959 | -0.8370 | -0.7039 | -0.7932 | -0.9216 ======= 0.9428 | 0.9398 | 0.9738 | 0.9679 | 0.9917 | 0.9817 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 2 | 300,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.4013 | -0.2231 | -0.2830 | -0.2135 | -0.3549 | -0.3199 | -0.6819 ======= 0.5933 | 0.4565 | 0.5157 | 0.6042 | 0.6072 | 0.6037 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 3 | 30,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.7634 | -0.7475 | -0.7374 | -0.6905 | -0.6600 | -0.7033 | -0.8664 ======= 0.7600 | 0.7123 | 0.7525 | 0.7959 | 0.8491 | 0.8233 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 4 | 300,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.2859 | -0.0602 | -0.1729 | -0.1374 | -0.2304 | -0.2131 | -0.5530 ======= 0.5477 | 0.3488 | 0.4929 | 0.5574 | 0.5481 | 0.5541 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 5 | 30,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD -0.8184 | -0.7769 | -0.7278 | -0.7638 | -0.6496 | -0.7286 | -0.8955 ======= 0.7826 | 0.7264 | 0.7520 | 0.8334 | 0.8364 | 0.8352 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 6 | 300,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD -0.7582 | -0.6891 | -0.5591 | -0.7297 | -0.6083 | -0.6904 | -0.8329 ======= 0.6689 | 0.5026 | 0.5693 | 0.7127 | 0.6950 | 0.7110 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 7 | 30,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD -0.7888 | -0.7142 | -0.6462 | -0.7525 | -0.6325 | -0.7150 | -0.8723 ======= 0.7009 | 0.6089 | 0.6316 | 0.7641 | 0.7575 | 0.7607 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 8 | 300,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD -0.7390 | -0.7006 | -0.5663 | -0.7322 | -0.4960 | -0.6632 | -0.8239 ======= 0.6451 | 0.4477 | 0.5598 | 0.6945 | 0.6699 | 0.6873 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 9 | 30,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.9602 | -0.9623 | -0.8920 | -0.8364 | -0.6981 | -0.7889 | -0.9228 ======= 0.9220 | 0.9200 | 0.9537 | 0.9547 | 0.9807 | 0.9701 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 10 | 300,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.5172 | -0.4540 | -0.4662 | -0.4573 | -0.5001 | -0.5093 | -0.7680 ======= 0.5986 | 0.4589 | 0.5221 | 0.6112 | 0.6140 | 0.6121 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 11 | 30,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.8003 | -0.8057 | -0.7681 | -0.7498 | -0.6698 | -0.7350 | -0.8929 ======= 0.7378 | 0.6852 | 0.7219 | 0.7805 | 0.8228 | 0.7987 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 12 | 300,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.3762 | -0.2532 | -0.3139 | -0.2712 | -0.3518 | -0.3419 | -0.6549 ======= 0.5486 | 0.3555 | 0.4992 | 0.5635 | 0.5543 | 0.5604 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 13 | 30,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD -0.8694 | -0.8520 | -0.7850 | -0.8045 | -0.6715 | -0.7598 | -0.9124 ======= 0.7926 | 0.7365 | 0.7639 | 0.8397 | 0.8444 | 0.8413 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 14 | 300,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD -0.7661 | -0.7077 | -0.5836 | -0.7417 | -0.6210 | -0.7032 | -0.8567 ======= 0.6651 | 0.5095 | 0.5754 | 0.7199 | 0.7069 | 0.7174 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 15 | 30,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD -0.8080 | -0.7669 | -0.6897 | -0.7596 | -0.6432 | -0.7231 | -0.8884 ======= 0.7153 | 0.6211 | 0.6414 | 0.7731 | 0.7680 | 0.7719 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 16 | 300,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD -0.7355 | -0.7030 | -0.5716 | -0.7424 | -0.4546 | -0.6598 | -0.8420 ======= 0.6454 | 0.4573 | 0.5743 | 0.6967 | 0.6748 | 0.6879 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 17 | 30,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.9483 | -0.9552 | -0.8843 | -0.8346 | -0.6971 | -0.7874 | -0.9221 ======= 0.9000 | 0.8965 | 0.9307 | 0.9439 | 0.9699 | 0.9588 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 18 | 300,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.5410 | -0.4831 | -0.4843 | -0.5187 | -0.5206 | -0.5474 | -0.7940 ======= 0.5996 | 0.4591 | 0.5278 | 0.6153 | 0.6189 | 0.6155 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 19 | 30,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.7916 | -0.7933 | -0.7542 | -0.7493 | -0.6628 | -0.7304 | -0.8932 ======= 0.7353 | 0.6724 | 0.7056 | 0.7751 | 0.8103 | 0.7917 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 20 | 300,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.4445 | -0.3419 | -0.3732 | -0.4101 | -0.4267 | -0.4454 | -0.7067 ======= 0.5472 | 0.3545 | 0.5042 | 0.5696 | 0.5598 | 0.5663 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 21 | 30,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD -0.8894 | -0.8731 | -0.8004 | -0.8196 | -0.6786 | -0.7710 | -0.9139 ======= 0.8169 | 0.7522 | 0.7828 | 0.8584 | 0.8602 | 0.8580 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 22 | 300,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD -0.8113 | -0.7658 | -0.6339 | -0.7680 | -0.6377 | -0.7245 | -0.8767 ======= 0.6839 | 0.5377 | 0.6043 | 0.7308 | 0.7205 | 0.7285 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 23 | 30,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD -0.8456 | -0.8061 | -0.7222 | -0.7801 | -0.6548 | -0.7385 | -0.8952 ======= 0.7254 | 0.6280 | 0.6521 | 0.7796 | 0.7731 | 0.7778 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 24 | 300,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD -0.7679 ======= 0.6469 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | -0.7470 | <<<<<<< HEAD -0.6121 | -0.7711 | -0.4363 | -0.6760 | -0.8655 ======= 0.5708 | 0.7021 | 0.6763 | 0.6887 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
\(~\) \(~\) \(~\) \(~\)
<<<<<<< HEADThe mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:
## EB FIQT BR cl1 cl2 cl3 rep
## -0.7241292 -0.6723208 -0.6218042 -0.6612917 -0.5691792 -0.6445125 -0.8355375
The methods are ranked according to the results above in ascending order:
## EB FIQT BR cl1 cl2 cl3 rep
## 2 3 6 4 7 5 1
=======
Fraction of significant SNPs with improved association estimates due to method implementation, using a significance threshold of \(5 \times 10^{-4}\):
\(~\) \(~\) \(~\) \(~\)
Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-4}\):
Here we investigate the 24 different scenarios under a bimodal distribution of effect sizes. In order to create a bimodal distribution, we simulate 50% of effect sizes of the true effect SNPs from a normal distribution centered at 0 while the other half are generated from a normal distribution with mean 2.5. As above, we first have a look at the expected number of significant SNPs and the expected proportion of those in which their association estimate is significantly exaggerated.
Running the code provided in nsig_prop_bias_100sim.R, we obtain the following results:
| Scenario | n_samples | h2 | prop_effect | S | n_sig 5e-8 | prop_bias 5e-8 | prop_x 5e-8 | mse 5e-8 | sd(n_sig) 5e-8 | sd(prop_bias) 5e-8 | sd(prop_x) 5e-8 | sd(mse) 5e-8 | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD 0.61 | 1.0000 | 0.9583 | 0.001874 | 0.803 | 0.0000 | 0.1729 | 0.003349 ======= -0.001857 | -0.001860 | -0.001746 | -0.001620 | -0.001363 | -0.001535 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 2 | 300,000 | 0.3 | 0.010 | -1 | 855.06 | <<<<<<< HEAD 0.7741 ======= -0.000012 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.0910 | <<<<<<< HEAD0.000016 ======= | -0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 19.640 | <<<<<<< HEAD0.0127 | 0.0090 | 0.000001 ======= | -0.000017 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||
| 3 | 30,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD 24.74 | 0.9968 | 0.4860 | 0.000474 | 5.181 | 0.0111 | 0.1190 | 0.000207 ======= -0.000903 | -0.000888 | -0.000878 | -0.000809 | -0.000780 | -0.000828 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 4 | 300,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD 2820.65 ======= -0.000009 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.6243 | <<<<<<< HEAD 0.0470 | 0.000014 ======= -0.000005 | -0.000003 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 27.505 | <<<<<<< HEAD0.0085 | 0.0043 | 0.000001 ======= | -0.000006 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||
| 5 | 30,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD 85.88 | 0.7709 | 0.0882 | 0.000161 | 5.689 | 0.0503 | 0.0294 | 0.000041 ======= -0.001190 | -0.001124 | -0.001053 | -0.001131 | -0.000955 | -0.001076 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 6 | 300,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD 574.45 ======= -0.000076 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.5513 | <<<<<<< HEAD 0.0337 | 0.000015 | 12.756 | 0.0207 | 0.0080 | 0.000002 ======= -0.000057 | -0.000073 | -0.000061 | -0.000069 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||
| 7 | 30,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD 281.92 | 0.6252 | 0.0475 | 0.000136 | 8.026 | 0.0287 | 0.0118 | 0.000022 ======= -0.000921 | -0.000843 | -0.000760 | -0.000888 | -0.000747 | -0.000844 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 8 | 300,000 | 0.8 | 0.001 | -1 | 729.09 | <<<<<<< HEAD 0.5264 | 0.0288 | 0.000015 | 12.631 | 0.0177 | 0.0056 | 0.000001 ======= -0.000062 | -0.000050 | -0.000065 | -0.000045 | -0.000059 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |
| 9 | 30,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD 0.44 | 1.0000 | 1.0000 | 0.002490 | 0.686 | 0.0000 | 0.0000 | 0.006906 ======= -0.001743 | -0.001749 | -0.001620 | -0.001533 | -0.001274 | -0.001444 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 10 | 300,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD 906.91 | 0.7750 | 0.0855 ======= -0.000025 | -0.000021 | -0.000022 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.000012 | <<<<<<< HEAD18.945 | 0.0132 | 0.0094 | 0.000001 ======= | -0.000024 | -0.000024 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |
| 11 | 30,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD 22.50 | 1.0000 | 0.5724 | 0.000392 | 4.520 | 0.0000 | 0.1058 | 0.000087 ======= -0.000863 | -0.000864 | -0.000824 | -0.000808 | -0.000722 | -0.000793 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 12 | 300,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD 2867.23 ======= -0.000012 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.6130 | 0.0442 | <<<<<<< HEAD0.000010 | 29.528 ======= | -0.000008 | -0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.0090 | 0.0038 | 0.000000 | |||
| 13 | 30,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD 91.79 | 0.7793 ======= -0.001328 | -0.001298 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.0867 | <<<<<<< HEAD0.000120 | 6.703 | 0.0456 | 0.0286 | 0.000026 ======= | -0.001221 | -0.001023 | -0.001156 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |
| 14 | 300,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD 543.36 | 0.5426 | 0.0315 | 0.000012 | 10.686 | 0.0230 | 0.0083 | 0.000001 ======= -0.000076 | -0.000071 | -0.000059 | -0.000074 | -0.000062 | -0.000070 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 15 | 30,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD 285.41 | 0.6093 | 0.0420 | 0.000101 | 9.313 | 0.0282 | 0.0111 | 0.000013 ======= -0.000974 | -0.000925 | -0.000830 | -0.000915 | -0.000772 | -0.000869 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 16 | 300,000 | 0.8 | 0.001 | 0 | 693.12 | <<<<<<< HEAD 0.5289 | 0.0307 ======= -0.000063 | -0.000051 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.000013 | <<<<<<< HEAD12.186 | 0.0176 | 0.0075 | 0.000001 ======= | -0.000045 | -0.000061 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||
| 17 | 30,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD 0.52 | 1.0000 | 1.0000 | 0.001402 | 0.717 | 0.0000 | 0.0000 | 0.001109 ======= -0.001642 | -0.001648 | -0.001527 | -0.001435 | -0.001202 | -0.001355 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 18 | 300,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD 918.79 | 0.8193 | 0.0945 | 0.000012 | 19.846 ======= -0.000026 | -0.000024 | -0.000023 | -0.000025 | -0.000025 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.0133 | 0.0096 | 0.000001 | |
| 19 | 30,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD 15.89 | 1.0000 | 0.7521 | 0.000474 | 3.408 | 0.0000 | 0.1188 | 0.000097 ======= -0.000803 | -0.000808 | -0.000767 | -0.000763 | -0.000676 | -0.000745 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 20 | 300,000 | 0.8 | 0.010 | 1 | 3130.86 | <<<<<<< HEAD 0.6014 ======= -0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.0392 | 0.000010 | <<<<<<< HEAD26.113 | 0.0098 | 0.0034 | 0.000000 ======= | -0.000013 | -0.000014 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||
| 21 | 30,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD 91.90 | 0.8221 | 0.0879 | 0.000119 | 5.901 | 0.0404 | 0.0293 | 0.000023 ======= -0.001310 | -0.001285 | -0.001181 | -0.001187 | -0.000990 | -0.001120 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 22 | 300,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD 521.36 | 0.5350 | 0.0309 | 0.000012 | 8.481 | 0.0212 | 0.0081 | 0.000001 ======= -0.000088 | -0.000082 | -0.000068 | -0.000083 | -0.000069 | -0.000078 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 23 | 30,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 313.57 | 0.5995 | 0.0392 | 0.000095 | 8.843 | 0.0282 | 0.0123 | 0.000010 ======= -0.001040 | -0.000990 | -0.000882 | -0.000970 | -0.000808 | -0.000915 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 24 | 300,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 628.44 | 0.5231 ======= -0.000071 | -0.000069 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.0298 | <<<<<<< HEAD0.000013 | 10.003 | 0.0184 | 0.0065 | 0.000001 ======= | -0.000071 | -0.000039 | -0.000062 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
\(~\) \(~\) \(~\) \(~\)
<<<<<<< HEADNext, we repeat the process illustrated in Section 2 using the third bias evaluation metric, rel_mse with a significance threshold of \(5 \times 10^{-8}\).
Summary of results for rel_mse contained in bimod_5e-8_10sim.csv:
Change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-4}\):
\(~\) \(~\) \(~\) \(~\)
Summary of results for rel_mse contained in norm_5e-4_10sim.csv:
| Scenario | n_samples | h2 | prop_effect | S | EB | FIQT | BR | cl1 | cl2 | cl3 | rep | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.8652 | -0.8611 | -0.8801 | -0.6651 | -0.7987 | -0.7563 | -0.8911 ======= -0.9547 | -0.9564 | -0.8976 | -0.8976 | -0.8976 | -0.8976 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 2 | 300,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.4005 | -0.0961 | -0.3535 | 2.3903 | 0.9330 | 1.5280 | -0.3260 ======= -0.4092 | -0.2321 | -0.2925 | -0.2925 | -0.2925 | -0.2925 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 3 | 30,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.8398 ======= -0.7544 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | -0.7968 | <<<<<<< HEAD -0.8554 | 0.4373 | -0.6085 | -0.2000 | -0.7343 ======= -0.7341 | -0.7341 | -0.7341 | -0.7341 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||
| 4 | 300,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.1200 | 0.2525 | -0.0535 | 2.2781 | 1.2585 | 1.6702 | -0.0908 ======= -0.2598 | -0.0230 | -0.1482 | -0.1482 | -0.1482 | -0.1482 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 5 | 30,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD -0.1251 | 0.4392 | 0.0816 | 2.4621 | 0.9733 | 1.5853 | -0.3879 ======= -0.8144 | -0.7689 | -0.7209 | -0.7209 | -0.7209 | -0.7209 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 6 | 300,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD 0.0368 | 0.4734 | 0.7842 | 1.2027 | 0.7414 | 0.9233 | 0.0476 ======= -0.7622 | -0.6931 | -0.5692 | -0.5692 | -0.5692 | -0.5692 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 7 | 30,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD -0.0272 | 0.9478 | 0.8110 | 2.5743 | 1.4722 | 1.9121 | -0.0455 ======= -0.7694 | -0.7048 | -0.6354 | -0.6354 | -0.6354 | -0.6354 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 8 | 300,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD 0.0758 | 0.2279 | 0.5819 | 0.6036 | 0.7481 | 0.5433 | 0.0105 ======= -0.7383 | -0.7019 | -0.5638 | -0.5638 | -0.5638 | -0.5638 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 9 | 30,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.8787 | -0.9023 | -0.8513 | -0.5169 | -0.9215 | -0.7961 | -0.8889 ======= -0.9592 | -0.9628 | -0.8915 | -0.8915 | -0.8915 | -0.8915 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 10 | 300,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.4038 | -0.0690 | -0.3713 | 2.4564 | 1.0098 | 1.5992 | -0.3064 ======= -0.5201 | -0.4514 | -0.4635 | -0.4635 | -0.4635 | -0.4635 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 11 | 30,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.8078 | -0.8759 | -0.8852 | 0.0434 | -0.7177 | -0.4219 | -0.7881 ======= -0.7985 | -0.8002 | -0.7631 | -0.7631 | -0.7631 | -0.7631 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 12 | 300,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.1118 | 0.2409 | -0.1032 | 2.0764 | 1.1539 | 1.5229 | -0.1016 ======= -0.3701 | -0.2461 | -0.3101 | -0.3101 | -0.3101 | -0.3101 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 13 | 30,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD -0.3081 | 0.6181 | 0.0185 | 3.1434 | 1.3302 | 2.0801 | -0.2942 ======= -0.8776 | -0.8574 | -0.7895 | -0.7895 | -0.7895 | -0.7895 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 14 | 300,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD 0.0274 | 0.3868 | 0.5348 | 1.1949 | 0.7106 | 0.9010 | 0.0196 ======= -0.7619 | -0.7069 | -0.5868 | -0.5868 | -0.5868 | -0.5868 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 15 | 30,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD -0.0590 | 0.6341 | 0.4830 | 1.8248 | 0.9890 | 1.3211 | -0.1607 ======= -0.8161 | -0.7741 | -0.6944 | -0.6944 | -0.6944 | -0.6944 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 16 | 300,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD 0.0406 | 0.3382 | 0.6162 | 1.1115 | 0.9371 | 0.9084 | 0.0302 ======= -0.7202 | -0.6922 | -0.5646 | -0.5646 | -0.5646 | -0.5646 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 17 | 30,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.9393 | -0.9454 | -0.9415 | -0.7304 | -0.9411 | -0.9054 | -0.8765 ======= -0.9499 | -0.9537 | -0.8838 | -0.8838 | -0.8838 | -0.8838 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 18 | 300,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.4866 | -0.2098 | -0.4695 | 2.7316 | 0.9980 | 1.7135 | -0.3306 ======= -0.5479 | -0.4947 | -0.4935 | -0.4935 | -0.4935 | -0.4935 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 19 | 30,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.8511 | -0.9174 | -0.9161 | -0.2153 | -0.7834 | -0.5711 | -0.8449 ======= -0.7875 | -0.7914 | -0.7520 | -0.7520 | -0.7520 | -0.7520 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 20 | 300,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.1137 | 0.3166 | -0.1154 | 2.1944 | 1.2935 | 1.6506 | -0.0561 ======= -0.4365 | -0.3315 | -0.3660 | -0.3660 | -0.3660 | -0.3660 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 21 | 30,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD -0.5084 | 0.0929 | -0.3487 | 2.3442 | 0.7355 | 1.3987 | -0.3590 ======= -0.8878 | -0.8709 | -0.8010 | -0.8010 | -0.8010 | -0.8010 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 22 | 300,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD 0.0575 | 0.3490 | 0.5646 | 1.2555 | 0.9525 | 0.9916 | -0.0665 ======= -0.8183 | -0.7712 | -0.6358 | -0.6358 | -0.6358 | -0.6358 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 23 | 30,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 0.0243 | 0.8020 | 0.4923 | 2.3288 | 1.3454 | 1.7375 | -0.0716 ======= -0.8484 | -0.8077 | -0.7194 | -0.7194 | -0.7194 | -0.7194 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 24 | 300,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 0.0481 | 0.2526 | 0.4966 | 0.6978 | 0.5453 | 0.5735 | 0.0543 ======= -0.7536 | -0.7355 | -0.6041 | -0.6041 | -0.6041 | -0.6041 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
\(~\) \(~\) \(~\) \(~\)
<<<<<<< HEADThe mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:
## EB FIQT BR cl1 cl2 cl3
## -0.31398333 0.02909167 -0.07000000 1.38432500 0.55651667 0.87122917
## rep
## -0.31077083
The methods are ranked according to the results above in ascending order:
## EB FIQT BR cl1 cl2 cl3 rep
## 1 4 3 7 5 6 2
\(~\) \(~\) \(~\) \(~\)
Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\) and a bimodal distribution of effect sizes:
\(~\) \(~\) \(~\) \(~\)
In this section, we proceed to comparing Winner’s Curse adjustment methods which use summary statistics from both the discovery and a replication GWAS. We first consider the situation in which the replication and discovery GWASs are of the same size. As an interesting comparison, we have also included the empirical Bayes method here which uses information from just the discovery GWAS. The significance threshold is set to 5e-8 and in this instance, the effect distribution is normal.
Summary of results for rel_mse contained in replicate_norm_5e-8_10sim.csv:
Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-4}\):
Here we investigate the 24 different scenarios under a bimodal distribution of effect sizes. In order to create a bimodal distribution, we simulate 50% of effect sizes of the true effect SNPs from a normal distribution centered at 0 while the other half are generated from a normal distribution with mean 2.5. As above, we first have a look at the expected number of significant SNPs and the expected proportion of those in which their association estimate is exaggerated.
Running the code provided in nsig_prop_bias_100sim.R, we obtain the following results:
| Scenario | n_samples | h2 | prop_effect | S | EB | rep | UMVCUE | cl1_com | cl2_MLE | cl3_MSE | MSE_min | MSE_min_sp |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | -0.6882 | -0.9846 | -0.7103 | -0.7103 | -0.9982 | -0.9833 | -0.9470 | -0.9532 |
| 2 | 300,000 | 0.3 | 0.010 | -1 | -0.3039 | -0.2882 | -0.2875 | -0.5715 | -0.5010 | -0.5468 | -0.4586 | -0.5775 |
| 3 | 30,000 | 0.8 | 0.010 | -1 | -0.6578 | -0.7883 | -0.7925 | -0.6863 | -0.8269 | -0.8061 | -0.7867 | -0.7768 |
| 4 | 300,000 | 0.8 | 0.010 | -1 | -0.0822 | -0.0425 | -0.0432 | -0.5127 | -0.4389 | -0.4863 | -0.2933 | -0.5106 |
| 5 | 30,000 | 0.3 | 0.001 | -1 | -0.1343 | -0.2324 | -0.2379 | -0.5933 | -0.4923 | -0.5561 | -0.4101 | -0.5825 |
| 6 | 300,000 | 0.3 | 0.001 | -1 | 0.0362 | 0.0439 | 0.0419 | -0.4850 | -0.4431 | -0.4703 | -0.2265 | -0.4209 |
| 7 | 30,000 | 0.8 | 0.001 | -1 | 0.0393 | -0.0825 | -0.0868 | -0.5366 | -0.4908 | -0.5311 | -0.3227 | -0.4751 |
| 8 | 300,000 | 0.8 | 0.001 | -1 | 0.0787 | -0.0052 | -0.0096 | -0.4909 | -0.4644 | -0.4833 | -0.2626 | -0.4641 |
| 9 | 30,000 | 0.3 | 0.010 | 0 | -0.6875 | -0.6995 | -0.6209 | -0.5615 | -0.8109 | -0.7351 | -0.6391 | -0.6923 |
| 10 | 300,000 | 0.3 | 0.010 | 0 | -0.2877 | -0.3129 | -0.3139 | -0.5805 | -0.5447 | -0.5762 | -0.4770 | -0.5850 |
| 11 | 30,000 | 0.8 | 0.010 | 0 | -0.6383 | -0.7218 | -0.7248 | -0.6759 | -0.7999 | -0.7777 | -0.7399 | -0.7385 |
| 12 | 300,000 | 0.8 | 0.010 | 0 | -0.0966 | -0.1277 | -0.1278 | -0.5285 | -0.4651 | -0.5063 | -0.3467 | -0.5265 |
| 13 | 30,000 | 0.3 | 0.001 | 0 | -0.1844 | -0.2980 | -0.3077 | -0.5853 | -0.5460 | -0.5770 | -0.4712 | -0.5594 |
| 14 | 300,000 | 0.3 | 0.001 | 0 | 0.0482 | 0.0126 | 0.0126 | -0.5011 | -0.4424 | -0.4783 | -0.2647 | -0.4252 |
| 15 | 30,000 | 0.8 | 0.001 | 0 | 0.0058 | -0.0662 | -0.0721 | -0.5415 | -0.4477 | -0.5004 | -0.3086 | -0.4029 |
| 16 | 300,000 | 0.8 | 0.001 | 0 | 0.1245 | -0.0364 | -0.0358 | -0.5072 | -0.4660 | -0.4906 | -0.2879 | -0.4689 |
| 17 | 30,000 | 0.3 | 0.010 | 1 | -0.6981 | -0.8833 | -0.8371 | -0.6669 | -0.9011 | -0.8732 | -0.8594 | -0.8607 |
| 18 | 300,000 | 0.3 | 0.010 | 1 | -0.2217 | -0.2487 | -0.2494 | -0.5630 | -0.5114 | -0.5477 | -0.4266 | -0.5653 |
| 19 | 30,000 | 0.8 | 0.010 | 1 | -0.5748 | -0.6226 | -0.6211 | -0.6454 | -0.7037 | -0.6968 | -0.6611 | -0.6794 |
| 20 | 300,000 | 0.8 | 0.010 | 1 | -0.0721 | -0.0740 | -0.0743 | -0.5209 | -0.4465 | -0.4912 | -0.3145 | -0.5166 |
| 21 | 30,000 | 0.3 | 0.001 | 1 | -0.1841 | -0.2900 | -0.2943 | -0.5556 | -0.5136 | -0.5407 | -0.4437 | -0.5233 |
| 22 | 300,000 | 0.3 | 0.001 | 1 | 0.0408 | -0.0108 | -0.0172 | -0.4975 | -0.4613 | -0.4878 | -0.2637 | -0.3899 |
| 23 | 30,000 | 0.8 | 0.001 | 1 | 0.0218 | -0.1438 | -0.1439 | -0.5176 | -0.4641 | -0.5037 | -0.3492 | -0.4009 |
| 24 | 300,000 | 0.8 | 0.001 | 1 | 0.1391 | -0.0311 | -0.0334 | -0.5134 | -0.4676 | -0.4952 | -0.2823 | -0.4587 |
\(~\) \(~\) \(~\) \(~\)
The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:
## EB rep UMVCUE cl1_com cl2_MLE cl3_MSE MSE_min
## -0.2073875 -0.2889167 -0.2744583 -0.5645167 -0.5686500 -0.5892167 -0.4517958
## MSE_min_sp
## -0.5647583
The methods are ranked according to the results above in ascending order:
## EB rep UMVCUE cl1_com cl2_MLE cl3_MSE MSE_min
## 8 6 7 4 2 1 5
## MSE_min_sp
## 3
\(~\) \(~\) \(~\) \(~\)
Relative change in average MSE over all significant SNPs due to method implementation with a replication and discovery GWAS of equal size, using a significance threshold of \(5 \times 10^{-8}\):
\(~\) \(~\) \(~\) \(~\)
We also look at how the methods compare when the replication data set is 50% that of the discovery data set.
Summary of results for rel_mse contained in replicate_norm_5e-8_10sim_halfrep.csv:
| Scenario | n_samples | h2 | prop_effect | S | EB | rep | UMVCUE | cl1_com | cl2_MLE | cl3_MSE | MSE_min | MSE_min_sp |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | -0.9465 | -0.8273 | -0.7601 | -0.5695 | -0.8224 | -0.8149 | -0.8362 | -0.8436 |
| 2 | 300,000 | 0.3 | 0.010 | -1 | -0.3233 | 0.5071 | 0.5072 | -0.3826 | -0.2016 | -0.2700 | 0.0454 | -0.3504 |
| 3 | 30,000 | 0.8 | 0.010 | -1 | -0.7190 | -0.2438 | -0.2946 | -0.4565 | -0.5059 | -0.4921 | -0.3910 | -0.5307 |
| 4 | 300,000 | 0.8 | 0.010 | -1 | -0.0825 | 0.8737 | 0.8728 | -0.3374 | -0.1311 | -0.2226 | 0.2818 | -0.2589 |
| 5 | 30,000 | 0.3 | 0.001 | -1 | -0.2625 | 0.5368 | 0.5098 | -0.3689 | -0.1362 | -0.2282 | 0.0930 | -0.2879 |
| 6 | 300,000 | 0.3 | 0.001 | -1 | 0.0100 | 0.8443 | 0.8435 | -0.3590 | -0.2866 | -0.3273 | 0.2495 | -0.2000 |
| 7 | 30,000 | 0.8 | 0.001 | -1 | 0.0173 | 0.9937 | 0.9918 | -0.3338 | -0.1174 | -0.2075 | 0.3843 | -0.0774 |
| 8 | 300,000 | 0.8 | 0.001 | -1 | 0.0353 | 0.9906 | 0.9703 | -0.3170 | -0.2633 | -0.2904 | 0.3297 | -0.1619 |
| 9 | 30,000 | 0.3 | 0.010 | 0 | -0.3697 | -0.8137 | -0.5393 | -0.4290 | -0.8910 | -0.8355 | -0.7105 | -0.7885 |
| 10 | 300,000 | 0.3 | 0.010 | 0 | -0.2771 | 0.4872 | 0.4864 | -0.3953 | -0.2142 | -0.2912 | 0.0418 | -0.3643 |
| 11 | 30,000 | 0.8 | 0.010 | 0 | -0.6187 | -0.3528 | -0.3591 | -0.4831 | -0.6165 | -0.5994 | -0.4701 | -0.5972 |
| 12 | 300,000 | 0.8 | 0.010 | 0 | -0.1012 | 0.7565 | 0.7555 | -0.3575 | -0.1833 | -0.2643 | 0.1983 | -0.2922 |
| 13 | 30,000 | 0.3 | 0.001 | 0 | -0.2337 | 0.5986 | 0.5843 | -0.4006 | -0.1511 | -0.2390 | 0.1228 | -0.2357 |
| 14 | 300,000 | 0.3 | 0.001 | 0 | 0.0483 | 0.9139 | 0.8996 | -0.3448 | -0.1797 | -0.2550 | 0.2947 | -0.1101 |
| 15 | 30,000 | 0.8 | 0.001 | 0 | -0.0499 | 0.7956 | 0.7857 | -0.3524 | -0.1500 | -0.2412 | 0.2286 | -0.0014 |
| 16 | 300,000 | 0.8 | 0.001 | 0 | 0.0733 | 1.1343 | 1.1239 | -0.3104 | -0.2131 | -0.2594 | 0.4253 | -0.1044 |
| 17 | 30,000 | 0.3 | 0.010 | 1 | -0.7573 | -0.7222 | -0.4972 | -0.4841 | -0.7932 | -0.7483 | -0.7040 | -0.7531 |
| 18 | 300,000 | 0.3 | 0.010 | 1 | -0.2005 | 0.6098 | 0.6091 | -0.3847 | -0.2005 | -0.2781 | 0.1110 | -0.3313 |
| 19 | 30,000 | 0.8 | 0.010 | 1 | -0.6205 | -0.2587 | -0.2740 | -0.4668 | -0.5206 | -0.5183 | -0.4250 | -0.5586 |
| 20 | 300,000 | 0.8 | 0.010 | 1 | -0.0677 | 0.8176 | 0.8163 | -0.3474 | -0.1834 | -0.2612 | 0.2395 | -0.2714 |
| 21 | 30,000 | 0.3 | 0.001 | 1 | -0.1625 | 0.6967 | 0.7020 | -0.3761 | -0.1921 | -0.2717 | 0.1407 | -0.1467 |
| 22 | 300,000 | 0.3 | 0.001 | 1 | 0.0335 | 0.9038 | 0.8923 | -0.3463 | -0.2270 | -0.2812 | 0.2960 | -0.0828 |
| 23 | 30,000 | 0.8 | 0.001 | 1 | 0.0525 | 0.7890 | 0.7675 | -0.3411 | -0.1669 | -0.2543 | 0.2224 | 0.0095 |
| 24 | 300,000 | 0.8 | 0.001 | 1 | 0.1719 | 1.0337 | 1.0202 | -0.3224 | -0.2172 | -0.2656 | 0.3748 | -0.1019 |
\(~\) \(~\) \(~\) \(~\)
The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:
## EB rep UMVCUE cl1_com cl2_MLE cl3_MSE
## -0.22293750 0.46101667 0.47557917 -0.38611250 -0.31517917 -0.36319583
## MSE_min MSE_min_sp
## 0.02261667 -0.31003750
The methods are ranked according to the results above in ascending order:
## EB rep UMVCUE cl1_com cl2_MLE cl3_MSE MSE_min
## 5 7 8 1 3 2 6
## MSE_min_sp
## 4
\(~\) \(~\) \(~\) \(~\)
Relative change in average MSE over all significant SNPs due to method implementation with a replication GWAS 50% of the size of the discovery GWAS, using a significance threshold of \(5 \times 10^{-8}\):
It is very interesting here how empirical Bayes behaves better in nearly all of the scenarios compared to UMVCUE, the replication method and the MSE minimization method.
\(~\) \(~\) \(~\) \(~\)
Finally, we look at how the methods compare when the replication data set is 10% that of the discovery data set.
Summary of results for rel_mse contained in replicate_norm_5e-8_10sim_10pc.csv:
| Scenario | n_samples | h2 | prop_effect | S | EB | rep | UMVCUE | cl1_com | cl2_MLE | cl3_MSE | MSE_min | MSE_min_sp |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | -0.9576 | -0.4613 | -0.1714 | -0.1714 | -0.8319 | -0.8711 | -0.5592 | -0.6053 |
| 2 | 300,000 | 0.3 | 0.010 | -1 | -0.3287 | 6.3266 | 6.3019 | -0.1162 | 0.8340 | 0.6565 | 3.4858 | 1.1209 |
| 3 | 30,000 | 0.8 | 0.010 | -1 | -0.6830 | 2.4395 | 2.2218 | -0.1057 | -0.0746 | -0.0838 | 1.2711 | 0.2963 |
| 4 | 300,000 | 0.8 | 0.010 | -1 | -0.0847 | 8.4091 | 8.4020 | -0.0973 | 0.7255 | 0.5296 | 4.7008 | 1.6199 |
| 5 | 30,000 | 0.3 | 0.001 | -1 | -0.2085 | 6.0848 | 5.8081 | -0.1262 | 0.9333 | 0.7615 | 3.3364 | 1.1594 |
| 6 | 300,000 | 0.3 | 0.001 | -1 | 0.0257 | 9.2150 | 9.0789 | -0.0912 | 0.2946 | 0.1921 | 5.2340 | 2.0676 |
| 7 | 30,000 | 0.8 | 0.001 | -1 | -0.0166 | 8.4894 | 8.3256 | -0.1246 | 0.8706 | 0.6712 | 4.7567 | 1.8498 |
| 8 | 300,000 | 0.8 | 0.001 | -1 | 0.0648 | 8.8348 | 8.5369 | -0.0897 | 0.1484 | 0.0826 | 4.9340 | 1.8612 |
| 9 | 30,000 | 0.3 | 0.010 | 0 | -0.7862 | 1.4177 | 0.7516 | -0.0651 | -0.4803 | -0.4362 | 0.8726 | 0.8886 |
| 10 | 300,000 | 0.3 | 0.010 | 0 | -0.2930 | 6.3992 | 6.3831 | -0.1067 | 0.7452 | 0.5794 | 3.5436 | 1.1743 |
| 11 | 30,000 | 0.8 | 0.010 | 0 | -0.6628 | 2.2452 | 2.0602 | -0.1431 | -0.0400 | -0.0622 | 1.1103 | 0.1811 |
| 12 | 300,000 | 0.8 | 0.010 | 0 | -0.1000 | 8.0019 | 7.9933 | -0.0949 | 0.7097 | 0.5303 | 4.4779 | 1.5427 |
| 13 | 30,000 | 0.3 | 0.001 | 0 | -0.1738 | 6.2345 | 5.9534 | -0.1046 | 0.8201 | 0.6574 | 3.4786 | 1.4050 |
| 14 | 300,000 | 0.3 | 0.001 | 0 | 0.0177 | 9.0005 | 8.8139 | -0.0930 | 0.4598 | 0.3378 | 5.1128 | 2.1003 |
| 15 | 30,000 | 0.8 | 0.001 | 0 | -0.0496 | 7.7050 | 7.5064 | -0.1070 | 0.5751 | 0.4182 | 4.3114 | 2.0258 |
| 16 | 300,000 | 0.8 | 0.001 | 0 | 0.1019 | 9.1618 | 8.9700 | -0.0813 | 0.1711 | 0.1034 | 5.1288 | 1.9984 |
| 17 | 30,000 | 0.3 | 0.010 | 1 | -0.7917 | 0.7464 | 0.8698 | -0.1488 | -0.3068 | -0.3178 | 0.2556 | 0.0458 |
| 18 | 300,000 | 0.3 | 0.010 | 1 | -0.2123 | 6.5887 | 6.5606 | -0.1087 | 0.7792 | 0.6088 | 3.6541 | 1.2120 |
| 19 | 30,000 | 0.8 | 0.010 | 1 | -0.6125 | 3.1323 | 3.1115 | -0.1485 | 0.0304 | -0.0062 | 1.7162 | 0.3105 |
| 20 | 300,000 | 0.8 | 0.010 | 1 | -0.0764 | 8.0191 | 7.9946 | -0.0993 | 0.6163 | 0.4489 | 4.4762 | 1.5375 |
| 21 | 30,000 | 0.3 | 0.001 | 1 | -0.0493 | 6.7867 | 6.5866 | -0.1167 | 0.9210 | 0.7058 | 3.7077 | 1.6689 |
| 22 | 300,000 | 0.3 | 0.001 | 1 | 0.0415 | 8.8260 | 8.5701 | -0.0936 | 0.3972 | 0.2748 | 4.9647 | 2.1855 |
| 23 | 30,000 | 0.8 | 0.001 | 1 | 0.0125 | 8.4717 | 8.2306 | -0.0925 | 0.6368 | 0.4645 | 4.7694 | 2.4695 |
| 24 | 300,000 | 0.8 | 0.001 | 1 | 0.1759 | 9.1156 | 8.8594 | -0.0802 | 0.2757 | 0.1787 | 5.1389 | 2.0793 |
\(~\) \(~\) \(~\) \(~\)
<<<<<<< HEADThe mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:
## EB rep UMVCUE cl1_com cl2_MLE cl3_MSE MSE_min
## -0.2352792 6.2995917 6.1549542 -0.1085958 0.3837667 0.2676750 3.4949333
## MSE_min_sp
## 1.3414583
The methods are ranked according to the results above in ascending order:
## EB rep UMVCUE cl1_com cl2_MLE cl3_MSE MSE_min
## 1 8 7 2 4 3 6
## MSE_min_sp
## 5
\(~\) \(~\) \(~\) \(~\)
Relative change in average MSE over all significant SNPs due to method implementation with a replication GWAS 10% of the size of the discovery GWAS, using a significance threshold of \(5 \times 10^{-8}\):
In this situation, the empirical Bayes method tends to behave better than all of the other methods.
\(~\) \(~\) \(~\) \(~\)
Under the same 24 scenarios, we have a look at simulating summary statistics for a binary trait and the corresponding performance of the various Winner’s Curse adjustment methods. Below is a summary of the results contained in binary_norm_nsig_prop_bias_5e-8_10sim.csv and binary_norm_nsig_prop_bias_5e-4_10sim.csv. Compared to the quantitative trait above, we witness a lot less SNPs meeting the significance threshold of 5e-8 under many of the scenarios.
Next, we repeat the process illustrated in Section 2 using the same bias evaluation metrics with a significance threshold of \(5 \times 10^{-8}\).
Summary of results for flb contained in bimod_5e-8_10sim.csv:
| Scenario | n_samples | h2 | prop_effect | S | n_sig 5e-8 | prop_bias 5e-8 | prop_x 5e-8 | mse 5e-8 | n_sig 5e-4 | prop_bias 5e-4 | prop_x 5e-4 | mse 5e-4 | sd(n_sig) 5e-8 | sd(prop_bias) 5e-8 | sd(prop_x) 5e-8 | sd(mse) 5e-8 | sd(n_sig) 5e-4 | sd(prop_bias) 5e-4 | sd(prop_x) 5e-4 | sd(mse) 5e-4 | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | 0.1 | 1.0000 | 1.0000 | 0.043205 | 503.2 | 1.0000 | 1.0000 | 0.024306 | 0.316 | NA | NA | NA | 19.384 | 0.0000 | 0.0000 | 0.001972 | |||||
| 2 | 300,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD 0.4 | 1.0000 | 1.0000 | 0.001337 | 563.2 | 1.0000 | 0.9590 | 0.002258 | 0.516 | 0.0000 | 0.0000 | 0.000347 | 23.748 | 0.0000 | 0.0107 | 0.000134 ======= 0.6451 | 0.4586 | 0.6244 | 0.5311 | 0.4978 | 0.5123 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 3 | 30,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD 2.3 | 1.0000 | 0.6667 | 0.018844 | 687.4 | 0.9960 | 0.8229 | 0.019493 | 1.337 | 0.0000 | 0.2976 | 0.009237 | 17.658 | 0.0028 | 0.0173 | 0.001161 ======= 0.9403 | 0.8540 | 0.9229 | 0.6343 | 0.8282 | 0.7446 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 4 | 300,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD 1353.9 | 0.7098 | 0.0732 | 0.000215 | 3870.3 | 0.7067 | 0.1751 | 0.000481 | 16.749 | 0.0106 | 0.0058 | 0.000011 | 44.672 | 0.0067 | 0.0044 | 0.000016 ======= 0.5674 | 0.2793 | 0.5434 | 0.5159 | 0.4857 | 0.4994 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 5 | 30,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD 0.2 | 1.0000 | 1.0000 | 0.023959 | 510.7 | 1.0000 | 0.9934 | 0.024185 | 0.422 | 0.0000 | 0.0000 | 0.011519 | 19.568 | 0.0000 | 0.0029 | 0.000956 ======= 0.5763 | 0.3771 | 0.5537 | 0.5583 | 0.5241 | 0.5396 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 6 | 300,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD 51.5 | 0.7965 | 0.1205 | 0.000298 | 713.3 | 0.9227 | 0.7224 | 0.001781 | 5.339 | 0.0565 | 0.0552 | 0.000110 | 28.960 | 0.0073 | 0.0099 | 0.000106 ======= 0.4930 | 0.1344 | 0.2900 | 0.5079 | 0.4801 | 0.5009 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 7 | 30,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD 130.2 | 0.7189 | 0.0761 | 0.002359 | 834.1 | 0.8651 | 0.6177 | 0.015121 | 7.657 | 0.0337 | 0.0238 | 0.000643 | 28.097 | 0.0080 | 0.0168 | 0.000689 ======= 0.5241 | 0.2150 | 0.3208 | 0.5136 | 0.4816 | 0.4960 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 8 | 300,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD 626.3 | 0.5347 | 0.0336 | 0.000186 | 1251.8 | 0.7170 | 0.4159 | 0.001057 | 17.982 | 0.0182 | 0.0052 | 0.000015 | 30.025 | 0.0117 | 0.0109 | 0.000059 ======= 0.4778 | 0.1009 | 0.2828 | 0.4994 | 0.4718 | 0.4882 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 9 | 30,000 | 0.3 | 0.010 | 0 | 0.1 | 1.0000 | 1.0000 | 0.031993 | 498.8 | 1.0000 | 1.0000 | 0.024378 | 0.316 | NA | NA | NA | 16.518 | 0.0000 | 0.0000 | 0.000919 | |||||
| 10 | 300,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD 0.3 | 1.0000 | 1.0000 | 0.001994 | 580.4 | 0.9998 | 0.9448 | 0.002149 | 0.483 | 0.0000 | 0.0000 | 0.002061 | 31.178 | 0.0005 | 0.0120 | 0.000140 ======= 0.6546 | 0.4648 | 0.6411 | 0.5354 | 0.4981 | 0.5120 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 11 | 30,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD 5.3 | 0.9833 | 0.4772 | 0.003954 | 718.3 | 0.9925 | 0.7883 | 0.017537 | 2.111 | 0.0527 | 0.2430 | 0.001587 | 25.755 | 0.0032 | 0.0111 | 0.000823 ======= 0.9465 | 0.8919 | 0.9176 | 0.6403 | 0.8035 | 0.7169 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 12 | 300,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD 1338.0 | 0.6829 | 0.0664 | 0.000120 | 3678.1 | 0.7050 | 0.1873 | 0.000463 | 22.376 | 0.0124 | 0.0047 | 0.000006 | 20.739 | 0.0059 | 0.0066 | 0.000024 ======= 0.5517 | 0.2668 | 0.5375 | 0.5100 | 0.4817 | 0.4939 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 13 | 30,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD 0.0 | NA | NA | NA | 514.1 | 1.0000 | 0.9941 | 0.023966 | 0.000 | NA | NA | NA | 11.930 | 0.0000 | 0.0033 | 0.000874 ======= 0.6741 | 0.3418 | 0.5541 | 0.5553 | 0.5044 | 0.5269 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 14 | 300,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD 56.4 | 0.7825 | 0.1075 | 0.000141 | 703.9 | 0.9209 | 0.7296 | 0.001785 | 5.661 | 0.0628 | 0.0393 | 0.000030 | 24.016 | 0.0068 | 0.0113 | 0.000134 ======= 0.4773 | 0.1266 | 0.2885 | 0.5025 | 0.4715 | 0.4954 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 15 | 30,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD 130.3 | 0.6866 | 0.0622 | 0.001186 | 816.3 | 0.8683 | 0.6312 | 0.015436 | 8.111 | 0.0497 | 0.0275 | 0.000160 | 16.180 | 0.0073 | 0.0160 | 0.000864 ======= 0.5144 | 0.1954 | 0.3006 | 0.5071 | 0.4853 | 0.4959 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 16 | 300,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD 585.6 | 0.5541 | 0.0330 | 0.000133 | 1218.8 | 0.7286 | 0.4246 | 0.001058 | 15.785 | 0.0179 | 0.0082 | 0.000013 | 20.115 | 0.0112 | 0.0155 | 0.000067 ======= 0.4806 | 0.1032 | 0.3029 | 0.5099 | 0.4783 | 0.4992 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 17 | 30,000 | 0.3 | 0.010 | 1 | 0.1 | 1.0000 | 1.0000 | 0.029079 | 505.2 | 1.0000 | 1.0000 | 0.024162 | 0.316 | NA | NA | NA | 15.259 | 0.0000 | 0.0000 | 0.001306 | |||||
| 18 | 300,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD 0.6 | 1.0000 | 0.8750 | 0.001202 | 580.7 | 0.9991 | 0.9277 | 0.002092 | 0.843 | 0.0000 | 0.2500 | 0.000601 | 20.907 | 0.0012 | 0.0087 | 0.000144 ======= 0.6937 | 0.5278 | 0.6818 | 0.5512 | 0.5114 | 0.5289 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 19 | 30,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD 8.0 | 0.9546 | 0.3713 | 0.003110 | 730.7 | 0.9851 | 0.7653 | 0.017206 | 2.582 | 0.0761 | 0.2686 | 0.001241 | 26.779 | 0.0045 | 0.0202 | 0.001488 ======= 0.9838 | 0.9449 | 0.9713 | 0.6567 | 0.9150 | 0.7789 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 20 | 300,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD 1343.7 | 0.6683 | 0.0574 | 0.000101 | 3445.6 | 0.6977 | 0.1899 | 0.000435 | 26.030 | 0.0139 | 0.0062 | 0.000003 | 42.717 | 0.0110 | 0.0075 | 0.000024 ======= 0.5555 | 0.2588 | 0.5421 | 0.5161 | 0.4858 | 0.4998 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 21 | 30,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD 0.1 | 1.0000 | 1.0000 | 0.028158 | 502.1 | 0.9998 | 0.9924 | 0.023958 | 0.316 | NA | NA | NA | 22.951 | 0.0006 | 0.0037 | 0.001363 ======= 0.6794 | 0.4069 | 0.6073 | 0.5699 | 0.5357 | 0.5488 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 22 | 300,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD 64.5 | 0.7587 | 0.0862 | 0.000115 | 697.8 | 0.9203 | 0.7345 | 0.001734 | 6.654 | 0.0399 | 0.0416 | 0.000032 | 28.142 | 0.0122 | 0.0132 | 0.000078 ======= 0.4896 | 0.1133 | 0.2642 | 0.5107 | 0.4895 | 0.5057 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 23 | 30,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 134.2 | 0.6752 | 0.0730 | 0.001053 | 793.5 | 0.8737 | 0.6537 | 0.016008 | 5.903 | 0.0244 | 0.0277 | 0.000162 | 21.629 | 0.0085 | 0.0167 | 0.000750 ======= 0.4903 | 0.1855 | 0.3245 | 0.5156 | 0.4776 | 0.4956 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 24 | 300,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD 532.7 | 0.5474 | 0.0316 | 0.000106 | 1159.1 | 0.7369 | 0.4396 | 0.001107 | 12.763 | 0.0103 | 0.0064 | 0.000010 | 18.806 | 0.0080 | 0.0085 | 0.000067 ======= 0.4819 | 0.0918 | 0.2496 | 0.5054 | 0.4851 | 0.4986 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
\(~\) \(~\) \(~\) \(~\)
<<<<<<< HEADWe plot \(z\) vs \(\text{bias}\) for the 24 different scenarios with a simulated binary trait. Similar to above, the points corresponding to SNPs which are significantly biased and are significant at a threshold of \(5 \times 10^{-4}\) are coloured in navy .
\(~\) \(~\) \(~\) \(~\)
\(~\) \(~\) \(~\) \(~\)
Summary of results for rel_mse contained in norm_5e-8_10sim.csv:
Fraction of significant SNPs with improved association estimates due to method implementation, using a significance threshold of \(5 \times 10^{-8}\) and a bimodal distribution of effect sizes:
\(~\) \(~\) \(~\) \(~\)
Summary of results for mse contained in bimod_5e-8_10sim.csv:
| Scenario | n_samples | h2 | prop_effect | S | EB | FIQT | BR | cl1 | cl2 | cl3 | rep | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.8321 | -0.7771 | -0.9121 | -0.6689 | -0.6791 | -0.6746 | -0.9954 ======= -0.001688 | -0.001791 | -0.001807 | -0.001527 | -0.001645 | -0.001620 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 2 | 300,000 | 0.3 | 0.010 | -1 | -0.9371 | -0.9499 | <<<<<<< HEAD -0.8904 | -0.7777 ======= -0.000006 | 0.000038 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | -0.9229 | <<<<<<< HEAD -0.8986 | -0.9533 ======= 0.000024 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||||
| 3 | 30,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.7689 | -0.7803 | -0.7013 | -0.0147 | -0.7514 | -0.4770 | -0.8230 ======= -0.000412 | -0.000448 | -0.000455 | 0.000056 | -0.000353 | -0.000196 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 4 | 300,000 | 0.8 | 0.010 | -1 | -0.2148 | <<<<<<< HEAD 0.1763 ======= 0.000004 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | -0.0974 | <<<<<<< HEAD 2.4029 | 1.1847 ======= 0.000029 | 0.000017 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 1.6734 | -0.2074 | ||||
| 5 | 30,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD -0.8933 | -0.8504 | -0.9085 | -0.9945 | -0.8166 | -0.9368 | -0.9954 ======= -0.000022 | 0.000036 | -0.000020 | 0.000376 | 0.000127 | 0.000229 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 6 | 300,000 | 0.3 | 0.001 | -1 | -0.2884 | 0.6545 | 0.4088 | 2.4080 | <<<<<<< HEAD 0.9838 | 1.5630 | -0.3425 ======= 0.000011 | 0.000013 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||||
| 7 | 30,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD -0.1623 | 1.0327 | 0.6538 | 3.7188 | 1.7049 | 2.5403 | 0.0034 ======= -0.000010 | 0.000087 | 0.000078 | 0.000272 | 0.000144 | 0.000194 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 8 | 300,000 | 0.8 | 0.001 | -1 | 0.0580 | <<<<<<< HEAD 0.3212 ======= 0.000004 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.6455 | <<<<<<< HEAD 0.7320 | 0.6382 | 0.6034 | -0.0108 ======= 0.000012 | 0.000013 | 0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||
| 9 | 30,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.8812 | -0.8610 | -0.9138 | -0.9951 | -0.8308 | -0.9421 | -0.9792 ======= -0.000764 | -0.000765 | -0.000752 | -0.000490 | -0.000768 | -0.000699 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 10 | 300,000 | 0.3 | 0.010 | 0 | -0.9555 | -0.9388 | -0.9438 | <<<<<<< HEAD -0.7604 | -0.8773 ======= 0.000029 | 0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | -0.8638 | -0.8946 | ||||
| 11 | 30,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.6630 | -0.6213 | -0.4645 | 0.6283 | -0.5625 | -0.0915 | -0.7332 ======= -0.000332 | -0.000359 | -0.000361 | -0.000005 | -0.000280 | -0.000172 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 12 | 300,000 | 0.8 | 0.010 | 0 | -0.2002 | <<<<<<< HEAD 0.1320 ======= 0.000003 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | -0.0877 | 2.1744 | 1.0360 | <<<<<<< HEAD 1.4948 | -0.2334 ======= 0.000015 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||||
| 13 | 30,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD NA ======= -0.000035 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | NA | <<<<<<< HEAD NA | NA | NA | NA | NA ======= -0.000009 | 0.000280 | 0.000122 | 0.000187 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||
| 14 | 300,000 | 0.3 | 0.001 | 0 | -0.2525 | 0.8169 | 0.5802 | 2.9517 | <<<<<<< HEAD 1.2249 | 1.9364 | -0.2569 ======= 0.000008 | 0.000010 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||||
| 15 | 30,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD -0.1714 | 0.6272 | 0.4434 | 2.2274 | 1.1211 | 1.5730 | -0.2541 ======= -0.000003 | 0.000067 | 0.000053 | 0.000218 | 0.000120 | 0.000159 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 16 | 300,000 | 0.8 | 0.001 | 0 | 0.0511 | 0.4392 | <<<<<<< HEAD 0.7611 | 1.2359 | 0.7964 | 0.9527 | 0.0005 ======= 0.000008 | 0.000014 | 0.000013 | 0.000012 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | ||
| 17 | 30,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.8904 | -0.8600 | -0.9145 | -0.9951 | -0.8296 | -0.9416 | -0.9994 ======= -0.001252 | -0.001142 | -0.001296 | -0.000599 | -0.000869 | -0.000742 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 18 | 300,000 | 0.3 | 0.010 | 1 | -0.6472 | -0.7991 | -0.4708 | -0.0551 | -0.8710 | -0.5756 | <<<<<<< HEAD -0.8783 ======= 0.000018 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||||
| 19 | 30,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.7983 | -0.7001 | -0.6627 | 0.2694 | -0.5722 | -0.2399 | -0.7207 ======= -0.000374 | -0.000397 | -0.000397 | -0.000054 | -0.000337 | -0.000228 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 20 | 300,000 | 0.8 | 0.010 | 1 | -0.1584 | 0.1997 | -0.0140 | <<<<<<< HEAD 2.1550 ======= 0.000020 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 1.0719 | <<<<<<< HEAD 1.5083 | -0.1636 ======= 0.000015 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |||||
| 21 | 30,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD -0.8899 | -0.8676 | -0.9156 | -0.9954 | -0.8391 | -0.9451 | -0.9961 ======= -0.000036 | 0.000021 | -0.000038 | 0.000289 | 0.000098 | 0.000177 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 22 | 300,000 | 0.3 | 0.001 | 1 | -0.1822 | <<<<<<< HEAD 0.8427 | 0.6334 | 2.7302 | 1.2514 | 1.8581 | -0.3071 ======= 0.000004 | 0.000006 | 0.000010 | 0.000006 | 0.000007 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | |
| 23 | 30,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD -0.0358 | 0.7604 | 0.5807 | 2.4714 | 1.1717 | 1.6978 | -0.0967 ======= 0.000009 | 0.000072 | 0.000042 | 0.000193 | 0.000168 | 0.000158 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 24 | 300,000 | 0.8 | 0.001 | 1 | 0.0415 | <<<<<<< HEAD 0.3314 ======= 0.000004 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.6490 | <<<<<<< HEAD 0.9037 ======= 0.000012 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd | 0.7276 | <<<<<<< HEAD 0.7336 | -0.0583 ======= 0.000009 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
\(~\) \(~\) \(~\) \(~\)
<<<<<<< HEADThe mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:
## EB FIQT BR cl1 cl2 cl3 rep
## -0.4640130 -0.1161478 -0.1539652 0.9022696 0.1895696 0.4586174 -0.5171957
The methods are ranked according to the results above in ascending order:
## EB FIQT BR cl1 cl2 cl3 rep
## 2 4 3 7 5 6 1
\(~\) \(~\) \(~\) \(~\)
Relative change in average MSE over all significant SNPs due to method implementation for a binary trait, using a significance threshold of \(5 \times 10^{-8}\):
\(~\) \(~\) \(~\) \(~\)
Summary of results for rel_mse contained in norm_5e-4_10sim.csv:
Change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\) and a bimodal distribution of effect sizes:
\(~\) \(~\) \(~\) \(~\)
Summary of results for rel_mse contained in bimod_5e-8_10sim.csv:
| Scenario | n_samples | h2 | prop_effect | S | EB | FIQT | BR | cl1 | cl2 | cl3 | rep | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 30,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.9941 | -0.9988 | -0.9147 | -0.8562 | -0.6989 | -0.7985 | -0.9300 ======= -0.8882 | -0.9082 | -0.9140 | -0.7189 | -0.8594 | -0.8248 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 2 | 300,000 | 0.3 | 0.010 | -1 | <<<<<<< HEAD -0.9753 | -0.9777 | -0.9085 | -0.8461 | -0.7023 | -0.7959 | -0.9218 ======= -0.3975 | -0.0841 | -0.3577 | 2.4220 | 0.9430 | 1.5479 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 3 | 30,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.9080 | -0.9117 | -0.8659 | -0.8112 | -0.6997 | -0.7788 | -0.9099 ======= -0.7891 | -0.8544 | -0.8707 | 0.1311 | -0.6646 | -0.3553 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 4 | 300,000 | 0.8 | 0.010 | -1 | <<<<<<< HEAD -0.3577 | -0.1615 | -0.2376 | -0.1789 | -0.3066 | -0.2761 | -0.6454 ======= -0.1208 | 0.2653 | -0.0497 | 2.1411 | 1.2413 | 1.5983 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 5 | 30,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD -0.9907 | -0.9943 | -0.9130 | -0.8427 | -0.6935 | -0.7882 | -0.9280 ======= -0.1081 | 0.3147 | -0.0549 | 2.3712 | 0.8515 | 1.4679 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 6 | 300,000 | 0.3 | 0.001 | -1 | <<<<<<< HEAD -0.8308 | -0.8050 | -0.7557 | -0.7759 | -0.6612 | -0.7417 | -0.9043 ======= 0.0300 | 0.4174 | 0.6944 | 1.2012 | 0.7558 | 0.9311 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 7 | 30,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD -0.8103 | -0.7525 | -0.7035 | -0.7686 | -0.6481 | -0.7314 | -0.8896 ======= -0.0637 | 0.6496 | 0.5894 | 1.9649 | 1.0608 | 1.4161 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 8 | 300,000 | 0.8 | 0.001 | -1 | <<<<<<< HEAD -0.7437 | -0.6824 | -0.5465 | -0.7248 | -0.5879 | -0.6805 | -0.8323 ======= 0.0520 | 0.2872 | 0.6181 | 0.7796 | 0.8747 | 0.7034 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 9 | 30,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.9954 | -0.9987 | -0.9134 | -0.8510 | -0.6963 | -0.7942 | -0.9299 ======= -0.9917 | -0.9925 | -0.9766 | -0.6280 | -0.9969 | -0.9042 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 10 | 300,000 | 0.3 | 0.010 | 0 | <<<<<<< HEAD -0.9755 | -0.9775 | -0.9024 | -0.8420 | -0.6984 | -0.7915 | -0.9275 ======= -0.3946 | -0.1075 | -0.3807 | 2.4194 | 0.9448 | 1.5473 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 11 | 30,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.9191 | -0.9307 | -0.8659 | -0.8246 | -0.6966 | -0.7831 | -0.9154 ======= -0.7912 | -0.8434 | -0.8545 | 0.0612 | -0.6428 | -0.3631 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 12 | 300,000 | 0.8 | 0.010 | 0 | <<<<<<< HEAD -0.4626 | -0.3698 | -0.3970 | -0.3819 | -0.4431 | -0.4437 | -0.7320 ======= -0.1084 | 0.2665 | -0.0957 | 2.0651 | 1.1895 | 1.5358 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 13 | 30,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD -0.9932 | -0.9965 | -0.9120 | -0.8503 | -0.6966 | -0.7943 | -0.9299 ======= -0.3183 | 0.5595 | -0.0743 | 2.5959 | 1.1429 | 1.7360 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 14 | 300,000 | 0.3 | 0.001 | 0 | <<<<<<< HEAD -0.8928 | -0.8802 | -0.8046 | -0.8170 | -0.6802 | -0.7708 | -0.9158 ======= 0.0285 | 0.3988 | 0.5762 | 1.1694 | 0.7158 | 0.8962 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 15 | 30,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD -0.8400 | -0.8168 | -0.7571 | -0.7792 | -0.6582 | -0.7405 | -0.9031 ======= -0.0257 | 0.6190 | 0.4922 | 1.9898 | 1.0994 | 1.4558 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 16 | 300,000 | 0.8 | 0.001 | 0 | <<<<<<< HEAD -0.7472 | -0.6972 | -0.5741 | -0.7391 | -0.6133 | -0.6978 | -0.8599 ======= 0.0493 | 0.3351 | 0.6395 | 1.1097 | 1.0027 | 0.9217 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 17 | 30,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.9962 | -0.9989 | -0.9136 | -0.8505 | -0.6966 | -0.7940 | -0.9294 ======= -0.9631 | -0.8789 | -0.9976 | -0.4609 | -0.6684 | -0.5710 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 18 | 300,000 | 0.3 | 0.010 | 1 | <<<<<<< HEAD -0.9709 | -0.9739 | -0.9010 | -0.8378 | -0.6965 | -0.7881 | -0.9234 ======= -0.4894 | -0.2587 | -0.4803 | 2.4502 | 0.8630 | 1.5173 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 19 | 30,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.9072 | -0.9181 | -0.8523 | -0.8179 | -0.6914 | -0.7770 | -0.9155 ======= -0.8711 | -0.9199 | -0.9222 | -0.0954 | -0.7784 | -0.5168 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 20 | 300,000 | 0.8 | 0.010 | 1 | <<<<<<< HEAD -0.5086 | -0.4340 | -0.4411 | -0.4843 | -0.4899 | -0.5149 | -0.7707 ======= -0.1136 | 0.3018 | -0.1147 | 2.1294 | 1.2529 | 1.5992 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 21 | 30,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD -0.9913 | -0.9961 | -0.9128 | -0.8582 | -0.7000 | -0.8001 | -0.9274 ======= -0.3030 | 0.1919 | -0.2896 | 2.3145 | 0.7983 | 1.4281 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 22 | 300,000 | 0.3 | 0.001 | 1 | <<<<<<< HEAD -0.8968 | -0.8857 | -0.8094 | -0.8129 | -0.6767 | -0.7661 | -0.9155 ======= 0.0525 | 0.3181 | 0.4888 | 0.7855 | 0.5018 | 0.6078 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 23 | 30,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD -0.8700 | -0.8444 | -0.7759 | -0.7960 | -0.6664 | -0.7523 | -0.9048 ======= 0.0926 | 0.7189 | 0.4250 | 1.9248 | 1.6475 | 1.5694 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
| 24 | 300,000 | 0.8 | 0.001 | 1 | <<<<<<< HEAD -0.7955 | -0.7539 | -0.6205 | -0.7730 | -0.6339 | -0.7268 | -0.8704 ======= 0.0418 | 0.3081 | 0.5540 | 0.8720 | 0.5590 | 0.6728 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd |
\(~\) \(~\) \(~\) \(~\)
<<<<<<< HEADThe mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:
## EB FIQT BR cl1 cl2 cl3 rep
## -0.8488708 -0.8231792 -0.7582708 -0.7550042 -0.6430125 -0.7219292 -0.8846625
The methods are ranked according to the results above in ascending order:
## EB FIQT BR cl1 cl2 cl3 rep
## 2 3 4 5 7 6 1
\(~\) \(~\) \(~\) \(~\)
Relative change in average MSE over all significant SNPs due to method implementation for a binary trait, using a significance threshold of \(5 \times 10^{-4}\):
Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\) and a bimodal distribution of effect sizes:
\(~\) \(~\) \(~\) \(~\)